There's no 'Count or Predict' but task-based \\selection for distributional models

In this paper, we investigate the differences between prediction-based (word2vec), dense countbased (GloVe) and sparse count-based (JoBimText) semantic models. We evaluate the models, which were selected because they can all be computed efficiently on large data, based on word similarity tasks and a semantic ranking task both for verbs and nouns. We demonstrate that prediction-based models yield higher scores than the other two models at determining a similarity score between two words. To the contrary, sparse count-based methods perform best in the ranking task. Further, sparse count-based methods benefit more from linguistically informed contexts, such as dependency relations. In summary, we highlight differences of popular distributional semantic representations and derive recommendations for their usage.